Interface method and system for genetic analysis data

ABSTRACT

Aspects of the present invention describe a method and apparatus for viewing and interfacing with gene expression on a computer device. A computer based device interfaces with gene the expression data by representing a plate having one or more wells and their contents with a matrix of elements addressable along at least two-dimensions. Each of the elements in the matrix are processed and then distinguished based upon their passing and failing one or more metrics for gene expression data. The interface displays those values associated with the distinguished elements in the matrix using colors, shapes and symbols. Depending on the results, some of the elements in the matrix may be flagged and then filtered from the data set. Remaining gene expression values not filtered from the matrix of elements are displayed and processed further.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims is related to Application No. 60/696,127, filedJun. 30, 2005 assigned to the assignee of the present inventionentitled, “Flagging and Filtering of Gene Expression and Genotyping” byYerramalli Subramaniam, Madhu Augustine, David Woo and Jacob Burghardt,which is incorporated herein by reference.

INTRODUCTION

Conventional gene expression results produced using real-time polymerasechain reaction (RT-PCR) generally requires extensive review byscientists before use in studies. During the review, scientists engagein the tedious and laborious process of eliminating poor data points,experimental failure and other items that may contribute to unreliableresults from the RT-PCR data. This process of quality-control or QC isan important aspect of the experiment as it eliminates non-usable datapoints and improves the reliability of the overall experiment orassociated study.

Unfortunately, larger experiments and sample sizes are rapidly making itcost prohibitive to perform meaningful QC manually. Increased well countof sample plates from 96-well plates to 384-well plates have increasedthe sheer volume of data collected by a factor of 4. Viewing the rawdata in numeric form can be difficult and lead to many errors.Consequently, the volume of information can no longer be feasiblyreviewed manually in a reasonable time or for acceptable costs. Futuresystems will undoubtedly result in even higher well-density on theplates and increased QC issues surrounding data collection.

Uniformity in implementing QC is also problematic for large amounts ofgene expression data. In these large experiments, QC responsibilitiesmay be divided and performed by a team of individuals rather than asingle scientist. Each person on the team may evaluate the geneexpression data and reach different conclusions. This may be the resultof viewing the raw or statistical data with different spreadsheets,browsers or numeric display tools. In some cases, one scientist maydiscard certain data gathered as inaccurate while another scientist mayuse a slightly different criteria and keep the results. In high volumethroughput systems this eventually may lead to either losing importantdata or conversely introducing other errors in the data collected.

Accordingly, it is desirable to improve the tools for displayinginformation and quality control for data derived from various types ofexperiments used in gene expression and other types of analysis.

BRIEF DESCRIPTION OF THE DRAWINGS

The skilled artisan will understand that the drawings, described below,are for illustration purposes only. The drawings are not intended tolimit the scope of the present teachings in any way.

FIG. 1 is a schematic illustrating a system for spectral detection andanalysis in accordance with various implementations of the presentinvention;

FIG. 2 is a schematic illustration of a system used for fluorescentsignal detection in accordance with various implementations of thepresent invention;

FIG. 3A is a flowchart diagram of operations to flag and filter geneexpression data as part of quality control (QC) in accordance withvarious implementations of the present invention;

FIG. 3B depicts a graphical user interface (GUI) for controlling thequality control (QC) metrics and the resulting flagging and filteringoperations they perform according to various implementations of thepresent invention;

FIG. 3C depicts a graphical user interface (GUI) to view flag andfiltering as performed in accordance with various implementations of thepresent invention;

FIG. 3D depicts an example graphical user interface displaying resultsassociated with multiple studies in accordance with variousimplementations of the present invention;

FIG. 3E provides a key to a sample set of example icons displayed on amatrix representation of a plate in accordance with aspects of thepresent invention;

FIG. 3F is another example table illustrating the use of one or more ofthe icons from key in FIG. 3D in accordance with aspects of the presentinvention;

FIG. 3G provides another example of a results table or QC summarydesigned in accordance with implementations of the present invention;

FIG. 3H depicts a table settings interface for specifying filteringlogic as applied to the various column values found in a results tablein accordance with aspects of the present invention;

FIG. 3I depicts the results of the QC metrics using a QC summary tablein accordance with aspects of the present invention;

FIG. 4 provides a flowchart of the operations for selecting anappropriate set of QC metrics in accordance with various implementationsof the present invention;

FIG. 5 depicts a flowchart diagram of the operations used to test forsensor saturation in accordance with various implementations of thepresent invention;

FIG. 6 depicts a flowchart for a QC metric using data measurements of apassive reference in accordance with various implementations of thepresent invention;

FIG. 7 is another flowchart for a QC metric that identifies one or moremissing data points in a sequence of measurements taken using a detectoraccording to various implementations of the present invention;

FIG. 8 depicts a flowchart of the operations for a QC metric designed toidentify an empty well in a plate according to various implementationsof the present invention;

FIG. 9A is another flowchart diagram depicting operations associatedwith a QC metric for determining a large mean square error (LMEvariation according to various implementations of the present invention;

FIG. 9B illustrates a measured composite signal or spectrum received bya detector and the individual dye components determined to havecontributed to the composite spectrum using multicomponent analysis inaccordance with various implementations of the present invention;

FIG. 9C illustrates an increase in fluorescence for individual dyecomponents as the thermal cycling advances over time according tovarious implementations of the present invention;

FIG. 10 is flowchart of the operations used by a QC metric measuring theoperation of the light source during an experiment in accordance withvarious implementations of the present invention;

FIG. 11 is a flowchart diagram for a QC metric that identifies whengenetic material and probe dyes in a well have not been amplifiedaccording to various implementations of the present invention;

FIG. 12 provides a flowchart diagram for a QC metric that identifiesnoise based on a high standard deviation of a threshold cycle (Ct) froma replicate group in accordance with various implementations of thepresent invention;

FIG. 13 depicts operations for a QC metric to measure a relative amountof noise in a given well in accordance with various implementations ofthe present invention;

FIG. 14 depicts a QC metric related to cluster analysis for allelicdiscrimination studies in accordance with various implementations of thepresent invention;

FIG. 15 depicts a flowchart and operations for relating the reliabilityof an allelic discrimination plot to the number of clusters found in theplot in accordance with various implementations of the presentinvention;

FIG. 16 depicts a flowchart of operations in a QC metric to ensure thevalidity of a given Ct is valid in accordance with variousimplementations of the present invention; and

FIG. 17 is a block diagram of a system used in operating an instrumentor method in accordance with various implementations of the presentinvention.

SUMMARY

Aspects of the present invention describe a method and apparatus forautomating quality control (QC) for gene expression data. Typically, acomputer based device receives gene expression data associated with atleast one spectral species and genetic sample in each well of a plate.Gene expression data may be received from a sequence detectioninstrument performing one or more gene expression related operations oneach of the wells of the plate. The computer based device identifiesgene expression data determined to have anomalous characteristicsaccording to a set of one or more QC metrics. The results from thesetests are used to conditionally flag wells of the plate affected by theanomalous characteristics according to various selected QC metrics.Filters can then be selectively applied to temporarily or permanentlyexclude the flagged data from subsequent gene expression studies.

Other aspects of the present invention describe a method of interfacingwith a system used to perform quality control on gene expression data.The interface may be displayed on a graphical user interface (GUI)running on a computer or computer-like device and receive controlcommands through a keyboard, mouse and other like devices. The interfaceused by the system graphically depicts a plate having one or more wellsand their contents with a matrix of elements addressable along at leasttwo-dimensions. Each of the one or more elements in the matrix isdistinguishable based upon the passing and or failing of one or more QCmetrics for gene expression data.

A combination of colors and shapes can be used to flag some elements inthe matrix as passing or failing certain QC metrics. For example, afirst geometric shape can be used to mark an element in the matrix whenthe well has passed all quality control metrics and a second geometricshape when the well has not passed all the quality control metricsspecified for the well. The interface also allows a user to controlfiltering of values associated with the distinguished one or moreelements in the matrix. Filtering may be applied selectively in responseto the one or more quality control metrics and to improve analysis ofthe gene expression data.

These and other features of the present teachings are set forth herein.

DESCRIPTION

FIG. 1 is a schematic illustrating a system for spectral detection andanalysis in accordance with some implementations of the presentinvention. System 100 includes a plate 102 with genetic samples, asequence detection instrument 104, a data collection computer 106, platedocuments 108, analysis session 110, studies 112 containing analyticalresults from many plates and a sequence data analysis computer 114. Toimprove the quality of information being processed, sequence dataanalysis computer 114 further utilizes one or more quality control (QC)metrics 116 to flag and filter gene expression data taken both from theplate documents 108 and the studies 112 repositories. In accordance withvarious implementations of the present invention, gene expression datainterface 118 presents the resulting gene expression data in anorganizational scheme and format that makes the information and patternseasier to comprehend, analyze and interpret.

Sequence detection instrument 104 includes a spectral detector capableof distinguishing certain spectral species emitted from the fluorescenceof reporter dyes interacting with the genetic material in wells on plate102. The spectra is typically monitored in real-time as a thermal cyclerin the sequence detection instrument 104 performs PCR on the geneticmaterial. For example, PCR operations may cause the sample or targetgenetic material to replicate and hybridize with increasing amounts of aSYBR green dye detectable in the wells of plate 102. After severalthermal cycles, the concentration of the target increases along with adetectable rapid increase of fluorescence from the SYBR green dye orother reaction substrate. A cycle threshold or Ct measurement is thenidentified when the measure of fluorescent intensity increases linearlyon a logarithmic scale compared with the increasing cycle number.Subsequent analysis of Ct values among various reactions may be used toidentify a concentration of the target genetic material.

Data collection computer 106 gathers raw data provided by sequencedetection instrument 104 and stores in plate documents 108 as requiredby a particular study or experiment being performed. The raw data islabeled, organized and stored by data collection computer 106 in one ofseveral different storage areas or files for subsequent processing. Forexample, the example in FIG. 1 depicts data collection computer 106 ascapable of storing the raw data in as plate documents 108 or studies112. In some cases, data collection computer 106 may also performcertain calibration operations or other types of basic data analysiswith the results to be stored in analysis sessions 110.

Resulting data stored in plate documents 108, studies 112 and analysissessions 110 are then made available to sequence data analysis computer114. Operations in sequence data analysis computer 114 not only mayperform further computational analysis but also perform and producequality control or QC metrics 116 on the stored data in order to improveoverall results. In particular, aspects of the present invention provideQC metrics 116 to help ensure the data used in the gene expressionanalysis has not been tainted by unforeseen events or equipmentmalfunctions. For example, some events affecting the gene expressionanalysis may include the physical failure of one or more wells in plate102, mismatched components in system 100, improper calibration or setupand many other causes.

In operation, system 100 uses QC metrics 116 to flag anomalous datagathered from plate 102 and then selectively filter or eliminate thedata. A user can setup various filtering conditions based upon theresults from QC metrics 116 and readily automate the QC process. In somecases, the filtering conditions can be triggered based on the detectablepresence of a certain condition while in other cases QC metrics 116 maycontinuously monitor other data and only filter certain data that meetsor exceeds certain filtering threshold requirements. For example, a setof wells in plate 102 may be detected and flagged by QC metrics 116 asmeeting nominal QC characteristics yet insufficient for filtering orelimination. Gene expression data interface 118 presents the resultinggene expression data in an organizational scheme and format using avisual representation of the underlying plate and wells. Specific shapesand colors are used to identify certain wells and samples meeting theaforementioned filtering and flag criteria. It is contemplated that useof gene expression data interface 118 alone or in combination with QCmetrics 116 to flag and filter improves the quality of the data analysisand may increase the throughput of analysis through automation. Both ofthese innovations are particularly important as well-densities of platesholding samples increase and sensors have increased sensitivity tosmaller variations in signal.

FIG. 2 is a schematic illustration of a system 200 used for fluorescentsignal detection in accordance with implementations of the presentinvention. This illustration depicts certain features typicallyassociated with the Applied Biosystems 7500 Real-Time PCR System.However, various aspects of the invention can be used in conjunctionwith the Applied Biosystems 7900HT Fast Real-Time PCR System model aswell as almost any other device involved with gathering and/or analyzingspectra from a sample.

Accordingly, detection system 200 illustrates some of the componentsmaking up the spectral detector and optics in sequence detectioninstrument 104 previously described in FIG. 1. Detection system 200 canbe used with real-time PCR (RT-PCR) processing in conjunction withaspects of the present invention. As illustrated, detection system 200includes a light source 202, a filter turret 204 with multiple filtercubes 206, a detector 208, a microwell tray 210 and well optics 212. Afirst filter cube 206A can include an excitation filter 214A, a beamsplitter 216A and an emission filter 218A corresponding to one spectralspecies selected from a set of spectrally distinguishable species to bedetected. A second filter cube 206B can include an excitation filter214B, a beam splitter 216B and an emission filter 218B corresponding toanother spectral species selected from the set of spectrallydistinguishable species to be detected.

Light source 202 can be a laser device, Halogen Lamp, arc lamp, OrganicLED, an LED lamp or other type of excitation source capable of emittinga spectra that interacts with spectral species to be detected by system200. In this illustrated example, light source 202 emits a broadspectrum of light filtered by either excitation filter 214A orexcitation filter 214B that passes through beam splitter 216A or beamsplitter 216B and onto microwell tray 210 containing one or morespectral species. Further information on light sources and overalloptical systems can found in U.S. Patent Application 20020192808entitled “Instrument for Monitoring Polymerase Chain Reaction of DNA”,by Gambini et al. and 200438390 entitled “Optical Instrument IncludingExcitation Source” by Boege et al. and assigned to the assignee of thepresent case.

Light emitted from light source 202 can be filtered through excitationfilter 214A, excitation filter 214B or other filters that correspondclosely to the one or more spectral species. These spectrallydistinguishable species may include one or more of FAM, SYBR Green, VIC,JOE, TAMRA, NED, CY-3, Texas Red, CY-5, Hex, ROX (passive reference) orany other fluorochromes that respond by emitting a detectable signal. Inresponse to light source 202, the target spectral species and selectedexcitation filter, beam splitter and emission filter combination providethe largest signal response while other spectral species with lesssignal in the band pass region of the filters contribute less signalresponse. Multicomponent analysis is typically used to determine theconcentration of the individual species according to their respectivecontribution to the emitted spectra.

Referring to FIG. 2, microwell tray 210 generally contains the genetictarget sample with one or more reporter dyes corresponding to the assayused in conjunction with an experiment. Microwell tray 210 can include asingle well or any number of wells however typical sets include96-wells, 384-wells and other well configurations. Of course,experiments may be designed to use many other plate configurationshaving different multiples of wells other than 96. The sample andparticular combination of dyes used in the selected assay may be sealedin microwell tray 210 using heat and an adhesive film to ensure they donot evaporate or become contaminated.

Detector 208 receives the signal emitted from spectral species inmicrowell tray 210 in response to light passing through theaforementioned filters. Detector 208 can be any device capable ofdetecting fluorescent light emitted from multiple spectrallydistinguishable species in the sample. For example, detector 208 can beselected from a set including a charge coupled device (CCD), a chargeinduction device (CID), a set of photomultiplier tubes (PMT),photodiodes and a CMOS device. Information gathered by detector 208 canbe processed in real-time in accordance with implementations of thepresent invention and through subsequent post-processing operations forperforming calibration and QC metrics in accordance with various aspectsof the present invention.

High throughput systems can use additional quality control processing toflag and or potentially eliminate anomalous data results. QC metricsdesigned in accordance with aspects of the present invention furtheranalyze patterns and conditions in the data to facilitate this process.As will be described in further detail later herein, the type ofexperiment and the assay used in the gene expression or genotypingoperation may influence the selection of QC metrics and the flagging andfiltering of data. To increase the usability of these QC metrics,templates can be established for the automatic flagging and filtering ofdifferent datasets depending on the type of experiment and the labprotocol established for the experiment. Each template can specify apredetermined combination of QC metrics for a study. Instead ofspecifying individual QC metrics, the study can specify a template ofpredetermined QC metrics. This can reduce human error and increaseuniformity in the data analysis portion of the study.

FIG. 3A is a flowchart diagram of operations to flag and filter geneexpression data as part of a quality control in accordance with variousimplementations of the present invention. It is understood that a singlepass of the flowchart diagram steps is provided even though one or moreof the steps may be performed in a different order, repeatedly or atleast until the QC metrics have filtered the anomalous data set values.Initially, the QC metrics receive gene expression data associated with aspectral species and genetic samples in each well of a plate (302). Aspreviously described, the gene expression data is typically the raw datacollected by the sequence detection system and data collection computerduring RT-PCR. In some cases, one or more calibration operations may beperformed on the gene expression data to compensate for systematic erroror other detectable peculiarities of the system.

Next, implementations of the present invention perform one or more geneexpression related operations on data from every well of the plate(304). It is contemplated that the one or more gene expression relatedoperations may encompass a wide-range of possibilities. In some cases,the type of operations may depend on the type of experiment beingperformed and the type of assay used to measure the resulting geneticquality. For example, one set of assays may be used for allelicdiscrimination experiments while absolute quantification experiments mayuse another. QC metrics may capture the results from a Ct determinationfor further consideration before the Ct value for a given reaction isactually generated.

QC metrics designed in accordance with aspects of the present inventionthen identify gene expression data determined to have anomalouscharacteristics (306). Some of the tests performed by the QC metricsinclude: sensor saturation analysis, bad passive reference analysis,missing data analysis, empty well analysis, large composite signal erroranalysis, low-laser power analysis, non-amplified well analysis,non-amplified plate analysis, a noise spike analysis, a high-relativenoise analysis, a distance between clusters analysis, Ct failureanalysis, a number of clusters analysis, exponential region failureanalysis, a Hardy-Weinberg analysis, a proportion of outliers analysis,a small sample number in cluster, baselining failure analysis, andthresholding failure analysis.

Each of these different tests provides an indicator as to the quality ofthe data collected from a well in plate or several plates in a study. QCmetrics generally flag one or more wells of the plate affected by theanomalous characteristics (308). If the data in a well meets severaltests from the QC metrics then several flags are associated with thewell for each of the tests. It is also possible that the wells andplates are part of different studies and therefore the plates andstudies may also be flagged with these tests results as well. In otherwords, a complete study may be flagged with multiple flags rather than asingle flag. This allows quality control data results to be consideredfor each well in a plate, each plate in a study or to compare one studywith another. In addition to these groupings, it is contemplated thatthe flags can be used in data-mining and exploring other possible trendsand systematic errors spanning a variety of different wells, plates andstudies potentially spanning across different datasets and experiments.

Next the QC metrics determine if it is appropriate to exclude selecteddata associated with flagged wells (310). The QC metrics can beprogrammed with default configuration settings to perform certain testsand then simply filter or exclude the flagged results. In some cases, anominal result from the QC metrics indicates that the data gathered isnot reliable and should be both flagged and filtered from further geneexpression operations (314). In these situations, the QC metrics aresimply looking for the presence of a certain qualitative condition inthe data to flag the well and then filter the results. For example, ifthe QC metrics determines that a well is empty then it would make senseto omit the results from the well as they would skew the gene expressioncalculations being performed on the other wells containing actualgenetic material and/or reporter dyes.

Once the flagged data in the well is excluded, the gene expressionrelated operations are performed again on the data from the remainingwells of the plate (316). By excluding the data deemed unreliable, it iscontemplated that the gene expression related operations will improveand have more accurate results. For example, excluding a well in a plateas lacking specified genetic material or being empty may result in amore accurate identification of a Ct or other important gene expressionvalues.

Other cases using the QC metrics may flag certain wells or studies froma test yet not filter the data, as the test results are consideredquantitatively insufficient for filtering (312). The configuration ofthe QC metrics in this latter case may be set up to operate using acontinuous measure of results and threshold values to indicate when toflag a results and when to both flag and filter a result from the data.It is also possible that the QC metrics is unable to identify even anominal result from the test and therefore may neither flag nor filterthe resulting well, plate and/or study. In either case, the QC metricswill generally indicate the wells that have been flagged but may onlyexclude certain data depending on the criteria for filtering orexcluding the data collected from the wells (312).

Referring to FIG. 3B, a graphical user interface (GUI) provides a methodof controlling the QC metrics and the resulting flagging and filteringoperations they perform in accordance with various implementations ofthe present invention. In this example implementation, a QC controlpanel 318 includes a list of QC metrics 320, QC flag-filter conditions322 and panel operators 324.

Each QC metric performed in this example is listed under QC metrics 320.However, it is contemplated that aspects of the present invention mayalso include greater or fewer QC metrics other than the onesspecifically listed as long as the analytic bears some relationship toidentification and removal of anomalous data from gene expressionoperations. The GUI in FIG. 3B can also be used in controlling manyother filtering operations for use in gene expression analysis even ifnot directly related to quality control.

In this example, there are ten (10) different QC metrics that may beperformed if it is deemed appropriate for the experiment and assay.While it is not indicated specifically, the QC metrics may be performedin the order they are listed or may be reordered in the event theanalysis results depends on the order or has other order-dependentcharacteristics or results.

QC flag-filter conditions initially may have default factory settings togive some basis for performing the flagging and filtering operations.Later, these QC flag-filter conditions 322 can be set individually andcustomized to the particular experiment, assay and quality controlgoals. Checking the “Flag Condition” box identifies the tests that theQC metrics will perform using the specified condition.

For example, “Large Mean Square” (LME) has the “Flag Condition” checkedand a “>” (greater than) condition with a value of 2000. Accordingly,this would mean that finding a LME value for a particular well greaterthan 2000 should result in filtering or excluding the data value fromsubsequent genetic expression operations and calculations. This helpsreduce further inaccuracies into the various gene expression operationsand calculations. In contrast, specifying no conditions requires only anominal value for the QC metric to be met and a flag to be set. Forexample, the QC metric labeled “A well has missing data (HMD)” requiresonly a nominal value for the QC metric to be met and therefore the flagfor a well is set as soon as the empty well condition is detected.

Unchecking or deselecting the “Flag condition” box for a particular QCmetric omits the test from being performed on the data. This may notonly reduce processing time but also ensures that data from certainwells in a plate are not inadvertently filtered or omitted if they arefound to meet the minimum conditions specified. In some alternateimplementations, the QC metric is still performed even when the flagcondition is unchecked but no filtering of data is performed. Instead,it is up to the operator of the detection system to manually remove theflagged data or ignore the flag and continue with the various geneexpression operations in the experiment.

FIG. 3C depicts a collection of GUT tools to view flag and filtering asperformed in accordance with various implementations of the presentinvention. In this example, flag and filter GUI 326 illustrates a matrix328 representing a plate, table settings 330 with characteristics foreach well and a results or QC summary 332 providing an overview of thequality control results. Matrix 328 contains one element for each wellin a plate addressable along two-dimensions—an x and y coordinate—andone or more flags set for the particular well.

Each QC metric is capable of independently flagging a well and thereforeit is possible that a single well may contain multiple or many differentflags. As illustrated, matrix 328 indicates that wells A6, A24, H15,I15, P6 and P24 have all failed one (“1”) QC metric test. It iscontemplated that multiple numeric values could be displayed for eachelement and well in the plate. For example, selecting or rolling overeach element in matrix 328 with a mouse or other pointer device producesa display in a tooltip or other graphical representation of the multipleQC metric values that the sample held in the corresponding well hadfailed.

It is also possible that matrix 328 represents an overall study and eachelement represents an aggregate of QC results for a complete plate.Numbers indicated on each element of matrix 328 may be used to providean aggregate of the flags identified for all the wells in the selectedplate in the study. Once again, by selecting an element in matrix 328 atool-tip or other interface device may indicate the list of flagscorresponding to the QC metrics associated with the plate in question.The operator of the computer analyzing the detection system results canconsider the QC metric values being displayed in subsequent portions ofthe analysis in the study.

If more detailed information is desired, the operator selects an elementin matrix 328 causing another matrix appear that represents one plate inthe study. This additional matrix then details each plate with wells andcorresponding flags as previously described. For example, an operator ofthe computer can single click a mouse-device over an element in a matrixrepresentation of a study to get a visual summary of the flags set andthen double-click the specific element to see an additional matrix witha detailed visual representation of the plate and corresponding wellswith flags.

Certain visual effects can also be used in conjunction with matrix 328to improve the identification and analysis of data. In large complexexperiments, it is important to present information in a consistent andrecognizable manner to decrease possible misreading of data andintroduction of possible operational errors. This is especially true inlaboratories with different people running experiments over many days,months and potentially years. Accordingly, a color system may beimplemented to identify an outcome of each QC metric and thecorresponding element of matrix 328. For example, a well passing all QCmetrics can be labeled with a “green” color while a “red” colorindicates that not only did the well not pass one or more QC metrics butthat the associated data was omitted or excluded. Further, a “yellow”color can be used to identify a well that has been flagged yet the datahas not been excluded. This may occur if the particular QC metric was tobe omitted from the testing or not associated with the given assay orexperiment at hand. Many other alternative color combinations can bederived and used in conjunction with aspects of the present invention asfurther described later herein.

Shapes can also be used in conjunction with colors to ensure thatinformation is conveyed accurately and quickly. A rectangular shape canbe used to indicate that a well has not been excluded while a triangularshape in the element on matrix 328 can indicate that the underlying datahas been excluded. This may be important if images of the matrix 328 areprinted on non-color printers or the person has difficultydistinguishing from one color or the other. It is contemplated that manyother combination of shapes and colors can be derived and used inconjunction with aspects of the present invention as further describedlater herein.

These color, shape and numeric details added to matrix 328 make the GUIeasier to use and reduces potential errors. For example, marking anelement in the matrix with a first geometric shape and a first color canbe used to indicate that the well has passed both specified andunspecified QC metrics. Meanwhile, marking an element in the matrix witha first geometric shape and a second color can be used to indicate thatthe well has passed the specified QC metrics but failed the unspecifiedQC metrics. Finally, marking an element in the matrix with a secondgeometric shape and a second color can be used to indicate that the wellhas failed to pass both specified and unspecified QC metrics. Ingeneral, it should be appreciated that associating QC metric resultswith various combinations of particular shapes, colors and numericvalues may reduce or eliminate human-error or other operational errorsotherwise present and significant in large scale and high-volumeexperiments that need to operate with very low error rates.

Table settings 330 provide a configurable display of results in eachwell as represented by the elements of matrix 328. In this example, thetable displays specified results for each well in a plate including:position (i.e., well position), flag (i.e., specific QC metrics detectedfor the well), sample name (i.e., an identifier or label that may beused to distinguish the well 100k copies, 10k copies, 10 copies aredifferent labels in this example), detector (i.e., specific detectordyes used in well), task (i.e., a particular task being performed), Ct(i.e., cycle threshold value detected for sample in well), Ct median(i.e., a median value of Ct used in statistical calculations) andquantity (i.e., an estimated absolute copy count). It is interesting tonote in this example that the well identified as “A6” is flagged asfailing a QC metric and as a result the sample count is not taken.Consequently, the task and Ct value identified for well position “A6”remains undetermined.

Results or QC summary 332 provides a sketch of the quality control for agiven plate and/or study. In this example, QC summary 322 provides a“Flag” heading with an acronym for the test a “Flag Name” heading with abrief description of the test along with a “Frequency” heading thatprovides a frequency of occurrence for the test in the plate and/orstudy and a “Location” heading listing the coordinates for the entriesin the plate passing the test. The operator of the detector system canuse this interface portion of the GUI to quickly identified trends andpotential problems as an experiment progresses. As a result of thisinformation, the operator can decide to continue the experiment re-runcertain experiments or eliminate certain plates from study and thencontinue performing gene expression or genotyping operations.

For example, an operator may use matrix 328 to visually see trends inthe data and identify systematic errors occurring the plate. A group ofentries in matrix 328 indicating empty wells may have resulted from arupture in one or more wells or other problems in the plate. In eitherevent, implementations of the present invention can be used quickly toidentify the source of these trends and either correct or re-runportions of the experiment.

FIG. 3D depicts an example graphical user interface displaying resultsassociated with multiple plates. Study results 334 illustrates ahigh-level summary of the flag and filtering for wells in multipleplates. In this example, a green marker in study results 334 indicatesall wells and plates have passed the specified QC metrics while theyellow marker indicates at least one flag was set in the plate but thatthe plate has yet to be excluded from the study. Plate results 336 canbe used to display gene expression statistics and values for individualplates in a given study. Flagging provided at the study level mayindicate certain characteristics of plates using plate level flags.These flags may indicate threshold levels associated with the overallplate rather than the individual wells as previously illustrated. Forexample, a plate level flag LPL indicates flagging of a “Laser PowerLow” condition and the NAP flag indicates flagging of a “Non AmplifiedPlate” condition in the identified studies.

FIG. 3E provides a key to a sample set of example icons displayed on amatrix representation of a plate in accordance with aspects of thepresent invention. In this example, a square or rectangular shape havinga green filled color represents that the sample in a particular well haspassed all the QC metrics specified during the quality control phase ofan experiment. No numeric value is specified in the green square orrectangular shape as the sample did not fail a specified or unspecifiedQC metric.

Next, the key in FIG. 3E includes a triangular shape having a yellowfilled color representing that the sample in the well has been flaggedby at least one QC metric. In this example, the sample contained in thewell associated with the yellow triangle failed three (“3”) different QCmetrics as indicated by the numeric value. Depending on theimplementation, the data associated with the well having the yellowfilled triangle may or may not be omitted from the data set used forsubsequent analysis. Accordingly, various aspects of the presentinvention may also include another icon or symbol to represent that thedata associated with a sample in a well has both been flagged andomitted from the data set. For example, the key in FIG. 3E illustrates arectangular or square shape filled with white and having a red linecolor to indicate that the sample data in the well has been both flaggedand omitted from the data set. Further, the key in FIG. 3E alsoconsiders that the user or operator may manually omit certain wells andsamples. For example, the key indicates that manually omitting the datafrom a sample in a particular well causes the corresponding well in thematrix to be marked with a red line formed into a rectangular or squareshape and a red cross or “X” pattern in the center. No numeric value isindicated in this latter icon as the manually omitted well and data isnot filtered due to a particular QC metric.

FIG. 3F is another example table illustrating the use of one or more ofthe icons from key in FIG. 3E. In this example, matrix 338 has a set ofwells E5, E9, D, 10, F10 and G8 that have been flagged and/or filtered.In particular, this example indicates that wells E5 and E9 containedsamples that failed to pass 2 different QC metrics and were filtered andomitted from further calculations. Alternatively, wells D10, F10, and G8were flagged for not passing 3 different QC metrics but were notfiltered from the data values under consideration during the analysis.

FIG. 3G provides another example of a results table or QC summarydesigned in accordance with implementations of the present invention. QCsummary in FIG. 3G provides a visual status indicator for each well inthe plate after analysis. For example, all the wells displayed in FIG.3G appear to have passed the QC metrics as indicated by the solid greenrectangle symbol appearing in the flag column. In general, each activeflag being used in the particular experiment or analysis has a column inthe results table. Different icons will appear in the flag column foreach respective sample depending on the results from the QC metric.

FIG. 3H depicts a table settings interface for specifying filteringlogic as applied to the various column values found in the results tableof FIG. 3H. The upper portion 340 of this interface specifies the orderand sequence of columns to display in the results table of FIG. 3H. Inthis example, a single flag “Flag <1>” is specified while the rest ofthe 10 entries are calculated or directly measured values. Lower portion342 of the table settings interface actual specifies the logic forconsidering the values in the various columns and flagging and/orfiltering the data as a result.

FIG. 3I depicts the results of the QC metrics using a QC summary table.The types of flags are listed and the frequency of their occurrence andmatrix location are identified. For example, it can be seen in thisexample that the flag LME for detecting Large Mean Square Error has beendetected at a relatively high frequency of 362 events. Conversely, theflags for LPL (“laser power low”) and CTF (“Ct Failure”) have occurredwith the lower frequencies of 1 and 3 respectively.

FIG. 4 provides a flowchart of the operations for selecting anappropriate set of QC metrics in accordance with various implementationsof the present invention. In many cases, the type of QC metrics used onthe results of an experiment depends on the type of experiment and theassays used to obtain the results. In this example, a detection systemprovides an interface for selecting general QC metrics for an overallgene expression analysis (402). General QC metrics are used for all thedifferent experiments and assays on a given detector system andtherefore are included for every QC operation. For example, oneimplementation may perform one or more general QC metrics that may testfor fluorescence off-scale (FOS), bad passive reference (BPR), emptywells (EW), has missing data (HMD) and large mean-square error (LME).Using these QC metrics as a baseline eliminates problems that may occurin many different situations. Further details on these tests aredescribed later herein.

Other QC metrics are selected specific to the type of experiment andassay used by the detector system. In one example implementation,performing allelic discrimination analysis (404) is associated withperforming QC metrics tailored to test results of gene expression forthese types of experiments. For example, QC metrics for allelicdiscrimination may be selected from a set of tests including: measuringa distance between clusters (DCN) with no template control (NTC), aHardy-Weinberg test (HW), a number of clusters (NOC) test, a percentageof outliers (POU) test and a small number of samples in cluster (SNS)test. Further details on these tests are provided later herein.

Yet another set of QC metrics are selected for testing results from bothabsolute and relative quantitation experiments (408). These may befurther divided into tests for absolute and relative quantitation ofgene expression (410) and tests tailored to related analysis failure forabsolute and relative quantitation (412). In the former case, absoluteand relative quantitation tailored to gene expression may include teststo identify conditions of: laser power low (LPL), non-amplified plate(NAP), non-amplified well (NAW), has noise spike (HNS) and high relativenoise (HRN). In the latter case, detecting possible analysis failures(412), may test for exponential region algorithm failures (EAF),baselining algorithm failures (BAF), Ct algorithm failures (CAF) andthresholding algorithm failures (TAF).

Many other types of tests and groupings of tests are possible dependingon the experiments performed and details related to the analysis of theresults. Accordingly, it is contemplated that a set of QC metrics aredeveloped in accordance with various implementations of the presentinvention to correspond to each different type of expression analysis.In general, a set of QC metrics can be developed for any gene expressionor genotyping by first identifying the type of gene expression analysisbeing performed (414) and then selecting a preferred set of QC metricssuited for testing the results of the gene expression analysis.

Specifically, a number of different QC metrics are described in furtherdetail below. These examples illustrate the scope and variety of thevarious types of tests performed in one or more set of circumstances.However, it is understood that these are merely illustrative examplesand aspects of the present invention should not be limited to theseexamples alone. Instead, it is contemplated that aspects of the presentinvention can use many different types of tests for QC metrics asdictated by the different gene expression experiments being performedand their results.

Referring to FIG. 5, a flowchart diagram depicts the operations used totest for sensor saturation in accordance with various implementations ofthe invention. While there may be many reasons for its occurrence,saturation generally occurs when the exposure time for a sensor is toolong or when an excess amount of sample and or dyes produce too muchsignal for the detector. To test for saturation, the QC metricidentifies a maximum signal response for a detector when applied to awell in a plate (502). This information is generally part of theconfiguration information for a detector and may be stored in a memoryon the detector device. Alternatively, a user may enter this informationregarding sensor sensitivity when configuring the overall QC metrics forthe detector.

Next, the QC metric for saturation compares each measured intensitylevel for each well to determine if it exceeds the maximum signalresponse for the sensor (504). If the intensity level does not exceedthe maximum signal response rated for the sensor then the QC metricindicates that the measured intensity level from fluorescent activity iswithin the range of sensor sensitivity (510). Keeping the fluorescentactivity within the sensors dynamic range ensures that measurementresults are accurate and not beyond a level the sensor can detect.

Conversely, the QC metric may also determine that the measured intensityexceeds the maximum signal response for a sensor (504). If this occurs,the QC metric indicates that the measured signal intensity may not beaccurate (506) and flags the data accordingly. Saturation is identifiedas intensity values that clearly exceed or come very close to a maximumsensitivity value for a sensor. For example, a signal measured asoperating at the maximum signal intensity detectable by the sensor mayactually be producing a greater intensity value that cannot be measured.Consequently, aspects of the present invention may flag the well and itscontents as beyond the measurement capabilities of the detector.Further, aspects of the present invention, may warn the user and suggestchecking for causes of the saturation including excess quantity offluorescent dye in one or more wells and/or an overly long exposuretimes for the detector (508).

Yet another QC metric uses data measurements of a passive reference asdepicted in the flowchart operations in FIG. 6. In this example, the QCmetric measures intensities of the passive reference at various timeintervals to identify potential well failures and other problems in aplate. Unlike the sample in a well, the passive reference during anytime interval should exhibit relatively constant intensity levels duringRT-PCR and other experiments. Evidence of erratic or unstable passivereference intensity can be used as a predictor that one or more wellshave failed and now are either empty or partially filled.

Aspects of the present invention therefore carefully monitors thepassive reference intensity changes as a thermocycler operates. Theresults are used to identify a failed well, a leaky well or otherphysical anomalies in the plate holding the samples. To identify a badpassive reference, a QC metric first isolates a passive referenceresponse in an amplitude plot for a well (602). Typically, this involvesextracting the signal waveform of the passive reference from amulticomponent plot if a passive reference, such as ROX, is mixed in thewell with other dyes for normalization and other purposes.

Next, the QC metric identifies a start intensity and a subsequentintensity for the passive reference after predetermined number ofthermal cycles (604). In some cases, the start intensity and subsequentintensity measurements are measured in the earlier thermal cycles whilein other cases they may be measured later in the process. Takingmultiple readings may be used to improve the success of this test. Forexample, a passive reference may not leak or collapse until multipleheat cycles of the thermal cycler have been applied.

Various implementations of the present invention compare a differentialbetween the measured start intensity and the subsequent intensity forthe passive reference with a predetermined threshold (606). For example,the predetermined threshold may be based upon selecting one or severalstandard deviations from a mean intensity value to derive thedifferential. When the differential measured is smaller or equal to thethreshold there is less likelihood that the well associated with themeasured data has failed or has some inherent problems (612).

Alternatively, if the differential for the passive reference intensityexceeds the predetermined threshold (606) then aspects of the presentinvention may flag the data associated with the particular well ashaving a bad passive reference and well. In addition, aspects of thepresent invention may also suggest elimination of data from one or morewells in the plate according to a pattern of contamination fromidentified well failures (610). In the case the well leaked into otherwells, a single failed well may spill into other wells requiring theflagging and filtering of more than the original well identified asfailing. One pattern of contamination may extend radially outward apredetermined distance from the failed well.

Another QC metric depicted in FIG. 7 identifies one or more missing datapoints in a sequence of measurements taken using a detector. This QCmetric sequences through each data point taken for each cycle for a wellin a plate and for multiple probe dyes and or a passive reference (702).The missing data point can be identified as a sudden drop offintensities for one or several data point values followed by return ofthe intensity measurements.

Next, the QC metric determines if the missing data point is likely dueto a timing issue in the detector and or system (704). Certain timingissues or other peculiarities may occur in the data due to mismatchedfirmware or software versions in the instrument, system or detector.These types of errors are generally hard to detect as they may by foundsurrounded by a set of otherwise acceptable data points measured by thedetector. Unfortunately, a missing data point that occurs at a criticalpoint in the amplitude curve during an experiment can skew thecalculations and produce erroneous results. For example, a missing datapoint that occurs at or near a Ct value for an assay in RT-PCR cangreatly affect the outcome of an experiment.

In this case, aspects of the present invention indicates that a well inthe plate has a missing value for the given cycle due to a certaincondition, system behavior, configuration or other predetermineddetectable combination (706). One possible resolution is to suggesteliminating the data point from further gene expression calculations toavoid possible erroneous analysis results. Alternatively, aspects of thepresent invention may synthesize missing data point or pointsstatistically from surrounding data points (708). Depending on thereliability of the synthesized data, a reliability factor can beassociated with the data set associated with the well. For example, if amissing data point can be synthesized with a 90% certainty then areliability factor of 90% may be associated with any subsequent geneexpression calculations using the data. Of course, aspects of thepresent invention may also provide an indication that the one more wellsdo not have missing data due to due to anomalous system behavior and/orconfiguration (710).

FIG. 8 depicts a flowchart of the operations for a QC metric designed toidentify an empty well in a plate. This test differs from the QC metricfor missing data points depicted in FIG. 7 as the empty well will createdata points with smaller or negligible values throughout an experiment.Plates may have empty wells when the plate is not properly prepared orwhen the improper type or quantity of genetic material is used. Inaddition, a well may also appear empty if the well in a plate leaks orotherwise becomes damaged during an experiment. Accordingly, this QCmetric sequences through each data point taken during each cycle lookingfor multiple low-valued data points for a well in a plate and formultiple probe dyes and or a passive reference (802).

Next the QC metric determines if data points are consistently missingfor multiple probes and over many cycles (804). To be certain, aspectsof the present invention look for a consistent drop in the intensityvalues rather than erratic changes in the intensities. For example, if awell is empty or missing a specific probe then the well should continueto not generate any signal response even after several cycles.

If a sequence of data points is missing, aspects of the presentinvention indicate that a well in the plate is most likely empty ordefective due to consistently low or negligible signal response for eachcycle throughout a run or experiment (806). Various aspects of thepresent invention may also suggest elimination of the data from thewells identified as empty as they might introduce error in the geneexpression operations performed subsequently (808). Conversely, if thedata measured for a well is not consistently missing or low-valued thenaspects of the present invention indicate that the tested wells in aplate are likely not empty or defective and the data results can be usedin subsequent gene expression analysis (810).

FIG. 9A is another flowchart diagram depicting operations associatedwith a QC metric for determining a large mean square error (LME)variation. Mean-squared error represents the error that can occur whenconverting from composite spectral response obtained from the detectorto multi-component data. Finding a large mean square (LME) variation canbe indicative of the detector instrument being improperly calibrated,specifying incorrect dyes in the configuration and/or a fluorescencemeasurements outside the capabilities of the detector instrument.

Initially, the QC metric for this condition receives a measuredcomposite signal response emitted from the probe dyes and passivereference in each well of the plate (902). Using Multicomponentanalysis, the QC metric generates a multicomponent plot of each expecteddye component as derived from the composite signal response (904).Generally, the dye component breakdown depends on the configurationsettings in the instrument that identifies the set of dyes being used inthe particular assay.

As an illustration of this condition, FIG. 9B depicts the measuredcomposite signal or spectrum and the individual dye componentscontributing to the composite spectrum derived using multicomponentanalysis. There is no mean-squared error (MSE) indicated in FIG. 9B asthe composite signal is initially divided into the individual dyecomponents and no comparisons can be made.

The QC metric in FIG. 9A reverses this operation and then determines anexpected composite signal response by combining the dye components(906). The expected amplification curve can be derived through arecombination of the individual components in view of the calibrationsettings and other assumptions for each dye in the assay. To identify adifferential, the QC metric in FIG. 9A computes a difference between ameasured composite signal response and an expected composite signalresponse as computed from the individual components (908); thedifference between the measured composite signal and expected compositesignal response values is identified as the mean-squared error (MSE)

FIG. 9C illustrates the increase in fluorescence for individual dyecomponents as the thermal cycling advances over time. If the detectorinstrument is calibrated and configured correctly with the proper dyecomponents, a mean-squared error (MSE) should remain relatively constantand small as exemplified in FIG. 9C. When a difference between theexpected composite signal response and the measured composite signalresponse does not exceed a predetermined threshold (910) then the QCmetric indicates that the detector has probably been setup andcalibrated properly (916). However, when the difference exceeds theacceptable threshold then the QC metric flags the well as potentiallyholding inaccurate data. The QC metric may indicate that the dyecomponents and calibrations specified in the setup for the well ormultiple wells of the plate are incorrect (912). A flag is associatedwith the well indicating that setup for the well in the plate may beinaccurate (e.g., the wrong set of dyes may have been specified) andneed correction. It is also possible that the measured composite signalwas off-scale causing a large error in the recreation of the expectedcomposite signal. Consequently, implementations of the invention maysuggest excluding data from the identified wells unless the correct dyecomponents could be specified and off-scale measurements shifted withinthe operating range of the detector (914)

FIG. 10 is another flowchart of the operations used by a QC metricmeasuring the operation of the light source during an experiment. The QCmetric for operation of the light source reviews power level data forpowering the light source on the instrument throughout the run of anexperiment (1002). It is presumed that a drop in power affects theintensity of the light source and fluorescence measurements associatedwith the light source. For example, a light source causing probe dyes inRT-PCR to fluoresce may produce inaccurate results if the power sourcefluctuates and the light source produces varying intensities of light.

Aspects of the present invention identify one or more points in timethat the power dropped below acceptable levels for driving the lightsource (1004). If no drop in power is detected then the QC metricindicates that the light source had received consistent and proper powerthroughout the experiment and not likely the cause of inaccuracies orerrors (1010). Conversely, detected drops in power are correlated withwell positions read from the plate as a possible cause of inaccuraciesor errors. The QC metric flags and indicates that experimental resultsgathered from these wells may be incorrect or inaccurate due tofluctuation in the intensity of the light source. The same QC metric mayalso suggest excluding data gathered from one or more of these wellsduring the time interval identified and potentially servicing the powersupply and or light source of the detector (1008).

FIG. 11 is a flowchart diagram of a QC metric that identifies whengenetic material and probe dyes in a well have not been amplified. Forexample, it is expected that genetic material combined with the probedyes during RT-PCR will exhibit increased or amplified intensities asthe thermocycler operates; failing to exhibit amplified intensities mayindicate problems with the test, equipment, samples or no presence oftarget sequence in sample.

In this case, the QC metric first fits a straight line to anamplification curve of a probe dye derived through multicomponentanalysis (1102). A baseline amplification curve is then obtained bysubtracting the fitted straight line from the probe dye amplificationcurve (1104). The baseline amplification curve is used by the subsequentoperations herein to identify when wells of samples have not beenamplified.

Next, a differential comparison is made between a maximum amplitude ofthe baseline amplification curve compared with a minimum amplitude ofthe baseline amplification curve (1106). If the difference between themaximum and minimum amplitude of the baseline amplification curve doesnot exceed a predetermined minimum threshold, there may be anamplification problem occurring in the particular well. For example, atest for low or no amplification in a well may use a metric conditionmeasured:

log(Max_(baseline)/Min_(baseline))≦1. In this case, the metric valueless than or equal to 1 is a nominal value that may indicate little orno amplification of the sample in the well. If this condition occurs,the QC metric indicates via a flag that the well in the plate with thegenetic material and probe dye may not have been amplified or suffersother related problems (1108). Additional wells in the plate may betested for no amplification (1110 and 1102) or the QC metric may finishthe no amplification analysis and indicate that one or more wells shouldbe excluded from gene expression analysis due to no amplification ofsample in wells of the plate (1112). This QC metric may also beconfigured to stop the quality control testing once at least one well isfound to have no amplification.

Conversely, the QC metric may use an alternate metric condition measuredby: log(Max_(baseline)/Min_(baseline))>1 to identify when samples in awell have been amplified. In this case, amplification of a target occurswhen the resulting metric is greater than 1 as the maximum value in thebaseline exceeds the minimum by a non-negligible amount. Once again,various implementations of the present invention may continue to testadditional wells in the plate for amplification (1110). Alternatively,the QC metric may finish without testing more wells. The QC metric mayindicate that genetic materials in some of the wells in a plate havebeen amplified and or study as well as suggesting exclusion of somewells from a plate and or study due to no detectable amplification ofthe sample (1112).

FIG. 12 provides a flowchart diagram for a QC metric that tests areplicate group of samples for consistent results. Tests on thereplicate group include taking multiple readings from the same sample orsamples. Inconsistent results from the replicate group helps determineif noise or other factors have rendered the sample data unreliable.

Initially, the QC metric receives an amplification curve of the probedies in the wells of a plate for a replicate group (1202). The replicategroup generally includes multiple readings of samples in a set of wells.Statistical calculations are performed to obtain Ct values for themembers of the replicate group. In one implementation, aspects of thepresent invention computes a mean, a standard deviation and otherstatistics useful in evaluating the cycle threshold (Ct) andamplification results (1204).

The QC metric for this test depends on the number of standard deviationsaway from the mean Ct value. When the standard deviation for Ctreplicate group exceeds a standard deviation threshold (1206) then theQC metric suggests excluding the samples associated with the replicategroup (1208). For example, the samples taken may not be reliable due toexcessive noise levels during the sampling and/or other types ofinterference making the resulting values unreliable. Identifyinginconsistent results in the replicate group helps ensure studies do notuse data tainted by noise and other types of time-varying interference.Alternatively, the QC metric may indicate that the samples beingmeasured should be kept as the values appear within an acceptablestatistical range of values in replicate group (1210). For example, asmall standard deviation among the Ct values in the replicate groupindicates consistency of the test results and the unlikelihood ofexcessive noise or other sources of error and or interference with thesample.

FIG. 13 depicts operations for a QC metric to measure a relative amountof noise in a given well relative to other wells in a plate. The sourceof this noise may result from background fluorescence inherent to theinstrument and environment surrounding the instrument. Generally, a QCmetric determines a standard deviation of noise in instrumentattributable to a background noise fluorescence (1302). This may beachieved by measuring the deviation of noise when a well is filled witha buffer or other non-fluorescent material.

Next, the QC metric generates a standard deviation of noise/backgroundfluorescence for the set of probe dyes in each well of the plate (1304).The QC metric computes a standard deviation and a mean value for the setof probe dyes in all wells of plate using the individual standarddeviations for each well (1306).

To determine if a sample should be used, the QC metric compares thestandard deviation for noise in a well with the standard deviation ofnoise in the overall plate (1308). When the standard deviation of noisein the well clearly exceeds the overall noise of the plate then the wellmay have unreliable sample and data. For example, if the standarddeviation of background fluorescence in a given well is eight-timesgreater than the standard deviation of the background fluorescence ofthe entire plate then there may issues with using data from the samplesin the well. QC metric may suggest excluding the given sample from thewell on plate due to a relatively high noise quality of data comparedwith the overall noise in the plate (1310). Alternatively, the QC metricmay indicate that the relative noise level for the individual wells donot exceed the threshold level when compared with noise on the plate andtherefore could be used in a study (1312).

Another QC metric depicted in the flowchart in FIG. 14 relates to acluster analysis for allelic discrimination studies. In general, theallelic discrimination analysis can be improved if unreliable resultscan be eliminated from the clusters. In this example, the QC metricidentifies samples landing near a “no template control” (NTC) cluster.Specifically, this QC metric identifies a distance between a sampleallele compared with an NTC cluster on the allelic discrimination plot(1402). This distance is important as samples closer to the NTC clustergenerally do not amplify well and tend to introduce unreliable data.Accordingly, if the sample is within a threshold distance of an NTCcluster on the allelic discrimination plot (1404) then the QC metricindicates the sample is not reliable (1406). The QC metric may furthersuggest excluding the data from the sample near the NTC cluster toimprove overall testing results (1408). Alternatively, the QC metric mayinstead indicate that the distance of the sample from the NTC clustershould not affected amplification results of an experiment for a plateand or study if it is not beyond this predetermined threshold amount(1410).

Different scientific results may be inferred depending on the number ofclusters found in the plot as provided by the operations in FIG. 15.Accordingly, a QC metric may first determine a number of clusters ofsamples on the allelic discrimination plot (1502). A typical Allelicdiscrimination plot indicate no clusters or several clusters dependingon the details of the actual experiment or study. Accordingly, this QCmetric may identify a number of different clusters visually andnumerically (1504). The number of clusters may be used for subsequentexperiments and analysis. For example, a QC metric indicating 3 clustersmay correspond to the presence of g both homozygous and heterozygousalleles

FIG. 16 depicts a flowchart of operations in a QC metric to ensure agiven Ct is valid. In this example, the QC metric performs automaticdetermination of Ct based upon an amplification of one or several probedyes (1602). Typically, multicomponent analysis is used if multiple dyesare combined in each well of plate. The QC metric checks if the baselineregion of the curves can be identified and with a sufficient degree ofcertainty (1604). If the baselines for different dyes cannot bedetermined than a suggestion is made that the Ct cannot be determinedreliably either. Next, an exponential region on or around the detectedCt value is determined. If the exponential region can be determined withsufficient certainty then an indication is made that automaticdetermination of Ct is possible (1610).

FIG. 17 is a block diagram of a system used in operating an instrumentor method in accordance with implementations of the present invention.System 1700 includes a memory 1702 to hold executing programs (typicallyrandom access memory (RAM) or read-only memory (ROM) such as Flash, adisplay interface 1704, a spectral detector interface 1706, a secondarystorage 1708, a network communication port 1710, and a processor 1712,operatively coupled together over an interconnect 1714.

Display interface 1704 allows presentation of information related tooperation and calibration of the instrument on an external monitor.Spectral detector interface 1706 contains circuitry to control operationof a spectral detector including duplex transmission of data inreal-time or in a batch operation. Secondary storage 1708 can containexperimental results and programs for long-term storage including one ormore QC storage areas, calibration factors and other data useful intesting, calibrating and performing quality control on data derived fromthe spectral detector. Network communication port 1710 transmits andreceives results and data over a network to other computer systems anddatabases. Processor 1712 executes the routines and modules contained inmemory 1702.

In the illustration, memory 1702 includes a QC metric component 1716, QCmetric flagging component 1718, QC metric filtering component 1720, QCmetric Interface component 1722 and a run-time system 1724. Run-timesystem 1724 manages system resources used when processing one or more ofthe previously mentioned modules. For example, run-time system 1724 canbe a general-purpose operating system, an embedded operating system or areal-time operating system or controller.

System 1700 can be preprogrammed, in ROM, for example, usingfield-programmable gate array (FPGA) technology or it can be programmed(and reprogrammed) by loading a program from another source (forexample, from a floppy disk, an ordinary disk drive, a CD-ROM, oranother computer). In addition, system 1700 can be implemented usingcustomized application specific integrated circuits (ASICs).

Having thus described various implementations and embodiments of thepresent invention, it should be noted by those skilled in the art thatthe disclosures are exemplary only and that various other alternatives,adaptations and modifications may be made within the scope of thepresent invention. For example, various implementations of the inventionare described as being used for gene expression however it iscontemplated that the processing, analysis and graphical user interfacedescribed can be used directly for or adapted for use in genotypingdata, allelic discrimination type studies as well as any other type ofbiological or genetic analysis.

Embodiments of the invention can be implemented in digital electroniccircuitry, or in computer hardware, firmware, software, or incombinations thereof. Apparatus of the invention can be implemented in acomputer program product tangibly embodied in a machine-readable storagedevice for execution by a programmable processor; and method steps ofthe invention can be performed by a programmable processor executing aprogram of instructions to perform functions of the invention byoperating on input data and generating output. The invention can beimplemented advantageously in one or more computer programs that areexecutable on a programmable system including at least one programmableprocessor coupled to receive data and instructions from, and to transmitdata and instructions to, a data storage system, at least one inputdevice, and at least one output device. Each computer program can beimplemented in a high-level procedural or object-oriented programminglanguage, or in assembly or machine language if desired; and in anycase, the language can be a compiled or interpreted language. Suitableprocessors include, by way of example, both general and special purposemicroprocessors. Generally, a processor will receive instructions anddata from a read-only memory and/or a random access memory. Generally, acomputer will include one or more mass storage devices for storing datafiles; such devices include magnetic disks, such as internal hard disksand removable disks; magneto-optical disks; and optical disks. Storagedevices suitable for tangibly embodying computer program instructionsand data include all forms of non-volatile memory, including by way ofexample semiconductor memory devices, such as EPROM, EEPROM, and flashmemory devices; magnetic disks such as internal hard disks and removabledisks; magneto-optical disks; and CD-ROM disks. Any of the foregoing canbe supplemented by, or incorporated in, ASICs.

Thus, the invention is not limited to the specific embodiments describedand illustrated above. Instead, the invention is construed according tothe claims that follow and the full scope of their equivalents thereof.

1. A computer implemented method of interfacing with genetic analysisdata, comprising; representing a plate having one or more wells andtheir contents with a matrix of elements addressable along at leasttwo-dimensions; distinguishing one or more elements in the matrix basedupon their passing and failing one or more metrics for genetic analysisdata; and filtering values associated with the distinguished one or moreelements in the matrix from a study in response to the one or moremetrics.
 2. The method of claim 1 wherein the filtering improvesanalysis of the genetic analysis data.
 3. The method of claim 1 whereinrepresenting the plate includes displaying a graphical representation ofthe matrix on a display device.
 4. The method of claim 1 whereinrepresenting the plate includes a voice system that calls out thedistinguishable one or more elements in the matrix.
 5. The method ofclaim 1 wherein the contents in the one or more wells is selected from aset including: a probe dye, a passive reference and a sample.
 6. Themethod of claim 1 wherein the distinguishing further comprises,associating an element in the matrix with a first geometric shape whenthe well has passed all metrics specified for the well; and associatinga second element in the matrix with a second geometric shape when thewell has not passed all the metrics specified for the well.
 7. Themethod of claim 1 further comprising: marking an element in the matrixwith a first geometric shape and a first color to indicate that the wellhas passed both specified and unspecified metrics; and marking anelement in the matrix with the first geometric shape and a second colorto indicate that the well passed the specified metrics and yet failedthe unspecified metrics.
 8. The method of claim 1 further comprising:marking an element in the matrix with a second geometric shape and thirdcolor when the well has not passed all the metrics specified for thewell.
 9. The method of claim 1 further comprising: marking an element inthe matrix with a first geometric shape and third color around theborder of the first geometric shape and a contrasting color to the thirdcolor as a fill color on the interior of the first geometric shape whenthe well has not passed all the metrics specified for the well.
 10. Themethod of claim 1 further comprising: labeling each element in thematrix with a numeric value indicating a total number metrics associatedwith the element in the matrix and determined to have failed.
 11. Themethod of claim 1 wherein an element in the matrix represents a portionof data from a set of datasets including: a value from a well in aplate, an aggregate of values from a plate in a study, an aggregate ofvalues from a study in an experiment and an aggregate of values from anexperiment in a clinical trial.
 12. The method of claim 1 wherein themetrics are designed for performing quality control to improvereliability of the data represented by an element in the matrix.
 13. Themethod of claim 1 wherein filtering values associated with thedistinguished one or more elements, further comprises: specifying anapplication of one or more metrics to values associated with the one ormore elements of the matrix; and omitting the values associated with theone or more elements from further genetic analysis depending on theresults from the application of the one or more metrics.
 14. The methodof claim 1 wherein filtering values associated with the distinguishedone or more elements, further comprises: specifying an application ofone or more metrics to values associated with the one or more elementsof the matrix; and including the values associated with the one or moreelements in further genetic analysis depending on the results from theapplication of the one or more metrics.
 15. The method of claim 13wherein the results from the application of the one or more metrics arecompared with a threshold value to decide if the values should beomitted.
 16. The method of claim 12 wherein the metrics for qualitycontrol further comprise: receiving gene expression data associated withat least one spectral species and genetic sample in each well of aplate; performing one or more gene expression related operations on dataderived for each of the wells of the plate; identifying gene expressiondata determined to have anomalous characteristics according to a set ofone or more quality control metrics; and flagging one or more wells ofthe plate affected by the anomalous characteristics according to qualitycontrol metrics.
 17. The method of claim 16 further comprising: flaggingone or more studies using the plate determined to have at least one ormore wells flagged for the anomalous characteristics according toquality control metrics.
 18. The method of claim 16 further comprising:determining whether to remove selected data associated with the flaggedwells based upon the metrics for quality control; excluding selectedflagged data from gene expression related operations and indicateflagged wells.
 19. The method of claim 16 further comprising: selectinga set of one or more quality control metrics for automating the qualitycontrol of the genetic analysis according to a type of assay andexperiment being performed.
 20. The method of claim 19 wherein the assayand experiment being performed is selected from a set including: allelicdiscrimination, absolute quantitation and relative quantitation.
 21. Themethod of claim 20, wherein the set of one or more metrics for qualitycontrol is selected from a set of quality control metrics including: asensor saturation analysis, a bad passive reference analysis, missingdata analysis, an empty well analysis, a large composite signal erroranalysis, a low laser power analysis, a non-amplified well analysis, anon-amplified plate analysis, a noise spike analysis, a high-relativenoise analysis, a distance between clusters analysis, Ct failureanalysis, a number of clusters analysis, exponential region failureanalysis, a Hardy Weinberg analysis, a proportion of outliers analysis,a small sample number in cluster, baselining failure analysis, andthresholding analysis.
 22. The method of claim 1 wherein the geneticanalysis may be selected from a set including: gene expression andgenotyping.
 23. A computer program product for interfacing with geneexpression data, tangibly stored on a computer-readable medium,comprising instructions operable to cause a programmable processor to:represent a plate having one or more wells and their contents with amatrix of elements addressable along at least two-dimensions;distinguish one or more elements in the matrix based upon their passingand failing one or more metrics for gene expression data; and filtervalues associated with the distinguished one or more elements in thematrix from a study in response to the one or more metrics.
 24. Anapparatus for interfacing with gene expression data, comprising; meansfor representing a plate having one or more wells and their contentswith a matrix of elements addressable along at least two-dimensions;means for distinguishing one or more elements in the matrix based upontheir passing and failing one or more metrics for gene expression data;and means for filtering values associated with the distinguished one ormore elements in the matrix from a study in response to the one or moremetrics.